This invention relates to an image processing method and an image processing apparatus. More specifically, the invention relates to the image processing method and the image processing apparatus employing that method for performing edge portion extraction so as to perform image processing operations such as sharpness enhancement, grain suppression and the like on a color image signal (image data) obtained by photoelectrically reading a color image or a subject, as well as the image processing method and the image processing apparatus employing that method for subjecting the obtained image data to specified image processing operations such as sharpness enhancement, grain suppression and the like to produce image data for output.
A new technology has recently been introduced and this is a printer that relies upon a digital color image reproduction system, where the color image recorded on a film such as a negative film, a reversal film, a color print or the like is read photoelectrically by a photoelectric conversion device such as a CCD or the like to obtain image signals corresponding to each of three primary colors, namely, red (R), green (G) and blue (B) and subsequently the obtained image signals are converted into digital image data signals which are then subjected to various image processing operations to be reproduced on recording materials such as color paper and the like and on display device such as a CRT (cathode ray tube) and the like. The printer operating on this digital color image reproducing system has been commercialized as a digital photoprinter.
In the digital photoprinter, color images can be reproduced in such a way that they have desired colors and gradations by subjecting obtained original image signals to image processing operations, even if the color images are recorded on such as the negative film, the reversal film or the color print under unfavorable photographing conditions such as under-exposure, over-exposure or the like. The color images recorded on the negative film, the reversal film, color prints and the like can optionally be reproduced as different color images in colors and gradations.
Examples of these image processing operations include a method of enhancing sharpness (sharpness enhancement) of images by subjecting the image data signals representing a given image to image processing operations and a grain suppression method for decreasing grainy appearance of images by suppressing the grains of the film. Prior to performing the sharpness enhancement or grain suppression, the edge portion or a grain portion (or flat portion) must be extracted. Various methods for extracting the edge portion and the like have been known.
An exemplary known method for extracting the edge portion is of a type that adopts template matching. In this method, templates comprising some form, for example, a 3xc3x973 matrix, are prepared with reference to a function distribution of pixel values of surrounding pixels of a pixel of interest to be decided which of the prepared template a given image is nearest to. Now, the pixel values of the pixel of interest and its surrounding pixels are set as xj (j=1, . . . , n); k pieces of templates are prepared; a value of the ith template is set as wji (j=1, . . . , n). A sum of products pi is calculated when i =1, . . . , k by the following equation:
Pi=xcexa3jwjixc2x7xj(xcexa3 is a sum when j=1, . . . , n)
It is decided that the template which gives the maximum of pi matches with the given image.
Consider, for example, a case where eight templates a1 to a8 as shown in FIG. 10 are given to the image composed of the pixel of interest and eight pieces of its surrounding pixels as shown in FIG. 11A. Each of these templates a1 to a8 corresponds to each of directions to edges as shown by arrows b1 to b8. Operators shown in such templates are called as xe2x80x98Sobelxe2x80x99 operators with which each of the sums of products p1 to p8 is calculated according to the above equation. The sum of products p1 of the template a1 is calculated as follows:
p1=1xc3x971+2xc3x972+1xc3x973+0xc3x979+0xc3x9710+0xc3x974+(xe2x88x921)xc3x978+(xe2x88x922)xc3x977+(xe2x88x921)xc3x975=xe2x88x9219
Other cumulated sums p2 to p8 corresponding to respective other templates a2 to a8 are calculated in the same manner. The results are as follows:
p2=xe2x88x928
p3=11
p4=20
p5=19
p6=8
p7=xe2x88x9211
p8=xe2x88x9220
As a result, p4 shows the maximum value so that the direction obliquely downward from right to left that the arrow b4 of the template a4 indicates shows the direction to the edge (density gradient, that is, direction of gradient) of the pixel of interest 10, as shown in FIG. 11B, and, in this case, the value 20 of the sum of products p4 shows the intensity of the gradient. In such way, the gradient of the pixel of interest (vector quantity) is calculated. Subsequently, with reference to pixels 1 and 5 that are adjacent to the pixel of interest 10 and positioned in the direction of 90 degrees from its gradient direction, each of images with a 3xc3x973 matrix around each of the pixels is calculated employing the above templates a1 to a8 in the same manner as above to obtain the gradient direction of each of the adjacent pixels 1 and 5. If at least one of the pixels 1 and 5 is in the direction of 45 degrees or less from the direction of the pixel of interest 10, the pixel of interest is decided as the edge portion, that is, having connectivity.
Though the conventional method for extracting the edge portion described above is of course capable of extracting the edge portion to some extent, it in cases detects the grain portion as the edge portion falsely, thereby causing deterioration of quality of a finished image.
As illustrated in FIG. 9A, an example shows that, besides white lines which were original edge portions, the grain portions were falsely decided as the edge portions so that a number of white grains have remained.
As already described, in recent years, the digital color image reproduction technique has been known that photoelectrically reads the image in a frame recorded on a photographic film by a reading sensor such as a CCD sensor or the like, subjects the thus read digital image data to image processing operations including enlargement, reduction and other various corrections and allows the image to be formed on a recording material by a laser beam modulated on the basis of the digital image data that has been processed imagewise.
In such technique that digitally reads the image in the frame by the reading sensor such as the CCD sensor or the like, following steps have been taken in order to realize an accurate reading of the image: the image in the frame is read preliminarily (so-called prescan is performed); reading conditions (for example, light quantity to be irradiated to the image in the frame, charge storage time of CCD or the like) are determined in accordance with density and the like of the image in the frame; and the image in the frame is read again under the thus determined reading conditions (so-called fine scan is performed).
At the time of the prescan, correction values to be adapted to various correction processing on color gradation, hypertone, hypersharpness, granularity and the like are each detected on the basis of the read image so that the image read by the fine scan is subjected to correction processing based on each of the thus detected correction values.
The granularity that is one of the objects to be subjected to the correction processing is referred to a rate of graininess or coarseness of the image on the film. Higher the granularity finer the image. The granularity is referred in one case to a rate when compared with that of a specified reference film and in another case to a relative rate within one image on the same film.
The granularity is one of the important factors to determine image quality. Granularity corrections are disclosed in, such as, Unexamined Published Japanese Patent Application (tokkai) No. 63-272173 where the exposure condition of each image in the frame of a negative color film is decided before negative/positive reversal processing is performed and thereafter a method for variably correcting the granularity corresponding to each image in the frame on the basis of the exposure condition of each image in the frame obtained by this decision has been performed.
Sharpness enhancement processing operations such as hypersharpness or the like is a processing for producing a sharp image having a forceful effect by means of enhancing an edge portion of the image. Moreover, this sharpness enhancement processing has been performed by the method where exposure conditions, for example, under-exposure or over-exposure, in each image in the frame on the negative color film is decided and thereafter sharpness enhancement rate, that is, a degree of the sharpness enhancement adapted to each image in the frame has been variably corrected on the basis of the result obtained by the above decision.
However, such correction method of the granularity and the degree of the sharpness enhancement has applied correction values that have been detected in accordance with the exposure condition, for example, under-exposure or over-exposure, of the image recorded in a frame to the whole image of the frame so that the method has a problem that some region in a frame has been overly corrected or otherwise insufficiently corrected.
The present invention has been accomplished under such circumstances and has an first object providing an image processing method and an image processing apparatus capable of raising extraction precision of an edge portion to a higher level than that in the prior art in order to precisely extract the edge portion, thereby increasing image quality.
A second object of the invention is to provide an image processing method and apparatus embodying the method that are capable of performing a minute correction of granularity and sharpness enhancement degree in accordance with the difference of granularities in an image recorded in one frame, or the difference of an edge portion and an flat portion (grainy portion) in the image recorded in one frame.
In order to attain the first object described above, a first aspect of the invention provides an image processing method for subjecting digital image data of color image signals to specified image processing, comprising the steps of: calculating gradients representing directions and intensities of a pixel of interest and its surrounding pixels from pixel values of the pixel of interest and its surrounding pixels of image composed of the color image signals that are to be subjected to image processing; storing the calculated gradients; calculating connectivity of the pixel of interest with its surrounding pixels from the directions of the stored gradients; calculating directivities of respective R, G and B of the pixel of interest by calculating gradients of respective R, G and B channels of the pixel of interest; and deciding whether the pixel of interest is an edge portion or not from the connectivity and the directivities of respective R, G and B to extract the edge portion.
Preferably, when the connectivity of the pixel of interest with its surrounding pixels is calculated, a direction of gradient of the pixel of interest and directions of gradients of nearby pixels of the pixel of interest are compared, and thereby, if the directions of a specified number of the nearby pixels agree with a specified condition, it is decided that the connectivity exists.
Preferably, when the connectivity of the pixel of interest with its surrounding pixels is calculated, if the directions of gradient of the adjacent pixels that are located on opposite sides of the pixel of interest in a direction of 90 degrees from the direction of gradient of the pixel of interest are within 45 degrees, it is decided that the connectivity exists.
In order to solve the second object described above in addition to the first object, it is preferable that the image processing method described above, further comprises the steps of: calculating at least one correction amount of a degree of sharpness enhancement and a granularity in each of a plurality of specified regions based on whether the plurality of the specified regions formed by dividing the image data include the edge portion or not; correcting at least one of the degree of the sharpness enhancement and the granularity of the image data based on the calculated correction amount of each specified region; and subjecting the image data to specified image processing including correction of at least one of the degree of the sharpness enhancement and the granularity to produce output image data.
It is also preferable that the above-described image processing apparatus, further comprises the step of calculating a correction amount of the granularity in each of a plurality of specified regions based on each density of the plurality of the specified regions formed by dividing the image data.
In order to solve the first object, a second aspect of the invention provides an image processing apparatus for subjecting digital image data of color image signals to specified image processing, comprises: calculating means for calculating gradients of a pixel of interest and its surrounding pixels that represent directions and intensities of an edge from pixel values of the pixel of interest and its surrounding pixels of image composed of the color image signals that are to be subjected to image processing; storing means for storing the calculated gradients; calculating means for calculating connectivity of the pixel of interest with its surrounding pixel from the directions of the stored gradients; calculating means for calculating directivities of respective R, G and B of the pixel of interest by calculating gradients of respective R, G and B channels of the pixel of interest; and deciding means for deciding whether the pixel of interest is an edge portion or not from the connectivity and the directivities of respective R, G and B, thereby edge portion extraction is performed.
Preferably, the calculating means for calculating the connectivity of the pixel of interest with its surrounding pixel is to compare a direction of gradient of the pixel of interest and directions of gradients of nearby pixels, and wherein, if the directions of a specified number of the nearby pixels agree with a specified condition, it is decided that the connectivity exists.
Preferably, the calculating means for calculating the connectivity of the pixel of interest with its surrounding pixels decides that the connectivity exists, if the directions of gradients of the adjacent pixels that are located on opposite sides of the pixel of interest in a direction of 90 degrees from the direction of the gradient of the pixel of interest are both within 45 degrees from the direction of the gradient of the pixel of interest.
In order to solve the second object in addition to the first object, it is preferable that the above-described image processing apparatus, further comprises: sharpness enhancement degree correction amount calculating means for calculating a correction amount of a degree of sharpness enhancement in each specified region, based on whether a plurality of specified regions formed by dividing the image data include the edge portion or not; and sharpness enhancement degree correction means for correcting the degree of the sharpness enhancement of the image data based on the calculated correction amount of each specified region by the sharpness enhancement degree correction amount calculating means, wherein the image data is subjected to specified image processing including correction of the degree of the sharpness enhancement to produce output image data.
It is also preferable that the above-described image processing apparatus, further comprises: granularity correction amount calculating means for calculating a correction amount of a granularity in each specified region based on each density of the plurality of the specified regions; and granularity correction means for correcting the granularity of the image data based on the correction amount of each specified region calculated by the granularity correction amount calculating means.
Preferably, the granularity correction amount calculating means further calculates a correction amount of the granularity in each specified region based on whether the plurality of the specified regions include the edge portion or not.
It is further preferable that the above-described image processing apparatus, further comprises: granularity correction amount calculating means for calculating a correction amount of a granularity in each specified region based on whether a plurality of specified regions formed by dividing the image data include the edge portion or not; and granularity correction means for correcting the granularity of the image data based on the correction amount of each specified region calculated by the granularity correction amount calculating means.
In each of the above aspects, it is preferable that the specified condition with which the directions of the specified number of the nearby pixels agree is a condition that 50% or more of the directions of the nearby pixels are within 45 degrees from the direction of the pixel of interest. Alternatively, it is more preferable that the specified condition with which the directions of the specified number of the nearby pixels agree is a condition that 60% or more, 70% or more, 80% or more, or 90% or more of the directions of the nearby pixels are within 45 degrees or 30 degrees from the direction of the pixel of interest.
Preferably, the nearby pixels are referred to any one of 1xc3x973 pixels, 1xc3x975 pixels, 3xc3x975 pixels and 3xc3x977 pixels excluding the pixel of interest placed in center of each of above pixel configurations where more pixels are arranged in the direction perpendicular to the direction of gradient of the pixel of interest than the direction of gradient.
Preferably, deciding conditions of the connectivity of the pixel including a way of selecting the nearby pixels, the specified number of the nearby pixels whose directions agree with the specified condition and the specified condition with which the directions of the specified number of the adjacent pixels agree are determined in accordance with at least one of an original kind, an original size, a print magnification, a camera kind and an exposure condition.
Preferably, the gradients of the pixel of interest and its surrounding pixels based on the color image signals are calculated based on a luminance signal of the color image signals.
In order to solve the above second object, a third aspect of the invention provides an image processing apparatus for photoelectrically reading an image recorded on a film or a subject so as to obtain image data which is subjected to specified image processing to produce output image data, comprises: granularity correction amount calculating means for calculating a correction amount of a granularity in each specified region based on each density of a plurality of specified regions formed by dividing the image data; and granularity correction means for correcting the granularity of the image data based on the correction amount of each specified region calculated by the granularity correction amount calculating means.
According to this aspect, granularity correction amount is not calculated on the basis of exposure condition of the image photographed in one frame (hereinafter called as image in one frame) as a whole; however, the image data of the image in one frame is divided into respective specified regions, the granularity correction amount calculating means calculates the granularity correction amount of each of the specified regions and the granularity correction means corrects the granularity of the image data on the basis of the obtained correction amount. Namely, granularity correction can be performed by taking the density of partial image (a plurality of densities) of the image in one frame into consideration so that a careful and minute correction of the granularity can be realized.
It should be noted that, in this aspect, the method for dividing the image data of the image in one frame into respective specified regions is not limited to any particular way and any appropriate methods such as dividing it equally in a checkered board pattern, dividing it in a honeycomb pattern, in a radial pattern, in a concentric circular pattern or in any combination thereof keeping the center of the image data in the centers of patterns, dividing it into a central part in which a principal subject is apt to be positioned with high possibility and the other part and the like can be optionally selected.
Preferably, the granularity correction means corrects the image data as a whole by selecting an optimal value from among correction amounts calculated on respective specified regions.
Namely, according to the invention, granularity correction can be performed in accordance with partial regions such as one specified region, a plurality of specified regions and the like in the image in one frame where the principal subject is recorded so that granularity is not set in an average way with the exposure condition of the frame as a whole, but the image data can be corrected to have the granularity suitable for a part of principal regions.
Preferably, the granularity correction means corrects image data in each specified region based on the correction amount corresponding to the respective specified region.
Namely, according to the invention, granularity correction is performed to each specified region on the basis of the calculated correction amount of each of specified regions formed by dividing the image in one frame so that different granularity correction in one frame can be realized thereby allowing a careful and minute granularity correction in accordance with the image in one frame to be performed.
Preferably, the granularity correction means corrects only the image data in a specified region placed at a previously predetermined position based on the correction amount calculated from the specified region at the previously predetermined position.
In other words, according to the invention, since the granularity correction in one specified region previously predetermined or a plurality of specified regions previously predetermined is calculated among previous specified regions obtained by dividing the image in one frame and thereafter only the image data on such one specified region previously predetermined or a plurality of predetermined specified regions is corrected, the granularity correction can be partly performed within one frame.
When a plurality of specified regions are selected, the image data can be corrected by means of any one of the following ways: the granularity correction amount of each of such specified regions is calculated to correct the image data corresponding to each of such specified regions; the granularity correction amount of each of such specified regions is calculated to correct the image data in a plurality of specified regions as a whole in accordance with most appropriate correction amount in the correction amounts; the granularity correction amount of a plurality of specified regions as a whole is calculated to correct the image data in a plurality of specified regions as a whole.
Therefore, for example, after the granularity correction of the whole image region in one frame has been. performed on the basis of exposure condition as has been done conventionally, only the specified region in either under-corrected condition or over-corrected condition is selected so that a further granularity correction can be performed whereupon a more careful and minute granularity correction can be realized.
It is of course possible that only the specified region at the specific position of the image in one frame before subjected to the granularity correction is selected to perform the granularity correction on the selected region only or the method of the granularity correction is changed in accordance with the quality of the image data which is to be subjected to the granularity correction.
Preferably, the specified region is a region composed of one pixel. Namely, the most careful and minute granularity correction can be performed by composing the specified region of one pixel.
It is preferable that the above-described image processing apparatus, further comprises principal subject extraction means for extracting an occupied region of a principal subject from the image data, wherein, as the specified region, the occupied region extracted by the principal subject extraction means or the other region than the occupied region is selected.
By taking the above arrangement, since correction amount of the principal part within the image can be appropriately obtained in accordance with the image photographed in each frame, the granularity correction can be performed in accordance with the image in a careful and minute manner.
Preferably, the specified region is a set of pixels having specified density obtained from a density cumulated histogram of the image data.
Namely, since granularity of a region with higher density in the image data is better than that of a region with lower density, local deterioration of granularity to be possibly caused by the density difference within the image data in one frame can be prevented by changing the granularity correction amount according to the density of the principal part within the image that is obtained by the density cumulated histogram of the image data.
Preferably, the density is a relative density difference from a base density of a film applied. Hereby density, appropriate granularity correction amount can be in any cases calculated even if a film with different film base density is employed.
Moreover, the granularity correction condition used as a reference differs between negative film and reversal film, among film sensitivities, and among film kinds. Therefore, it is also preferable that the aforementioned image processing apparatus further comprises film kind distinguishing means for distinguishing a film kind, wherein the granularity correction amount calculating means adjusts a reference value for calculating the correction amount in accordance with the film kind provided from the film kind distinguishing means. By this arrangement, the granularity correction condition as the reference can be adjusted in accordance with the film kind which is to be subjected to the correction processing so that the granularity correction can be performed in a more careful and minute manner.
It is further preferable that the aforementioned image processing apparatus further comprises: sharpness enhancement degree correction amount calculating means for calculating the correction amount of a degree of sharpness enhancement in each specified region based on whether the plurality of specified regions formed by dividing the image data include an edge portion or not; and sharpness enhancement degree correction means for correcting the degree of the sharpness enhancement of the image data based on the correction amount of each specified region calculated by the sharpness enhancement degree correction amount calculating means.
Preferably, the granularity correction amount calculating means further calculates the correction amount of the granularity in each specified region based on whether the plurality of the specified regions formed by dividing the image data include an edge portion or not.
In order to solve the above second object, a forth aspect of the invention provides an image processing apparatus for photoelectrically reading an image recorded on a film or a subject so as to obtain image data which is subjected to specified image processing to produce output image data, comprises: sharpness enhancement degree correction amount calculating means for calculating a correction amount of a degree of sharpness enhancement in each specified region based on whether a plurality of specified regions formed by dividing the image data include an edge portion or not; and sharpness enhancement degree correction means for correcting the degree of the sharpness enhancement of the image data based on the correction amount of each specified region calculated by the sharpness enhancement degree correction amount calculating means.
In order to solve the above second object, a fifth aspect of the invention provides an image processing method for photoelectrically reading an image recorded on a film or a subject so as to obtain image data which is subjected to specified image processing to produce output image data, comprises the step of: calculating at least one of a correction amount of a granularity in each specified region based on each density of a plurality of the specified regions formed by dividing the image data and a correction amount of a degree of a sharpness enhancement in each specified region based on whether the plurality of the specified regions formed by dividing the image data include an edge portion or not, wherein the granularity or the degree of the sharpness enhancement of the image data is corrected based on the calculated correction amount of each region.
Preferably, the correction amount of the granularity in each of the specified regions is further calculated based on whether the plurality of the specified regions include the edge portion or not.
Further scope of applicability of the present invention will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit and scope of the invention will become apparent to those skilled in the art from this detailed description.